Tree crown delineation using detectreeRGB
Contents
Tree crown delineation using detectreeRGB¶
Forest Modelling
Context¶
Purpose¶
Accurately delineating trees using detectron2, a library that provides state-of-the-art deep learning detection and segmentation algorithms.
Modelling approach¶
An established deep learning model, Mask R-CNN was deployed from detectron2 library to delineate tree crowns accurately. A pre-trained model, named detectreeRGB, is provided to predict the location and extent of tree crowns from a top-down RGB image, captured by drone, aircraft or satellite. detectreeRGB was implemented in python 3.8 using pytorch v1.7.1 and detectron2 v0.5. Further details can be found in the repository documentation.
Highlights¶
detectreeRGB advances the state-of-the-art in tree identification from RGB images by delineating exactly the extent of the tree crown.
We demonstrate how to apply the pretrained model to a sample image fetched from a Zenodo repository.
Our pre-trained model was developed using aircraft images of tropical forests in Malaysia.
The model can be further trained using the user’s own images.
Contributions¶
Notebook¶
Modelling codebase¶
Modelling funding¶
The project was supported by the UKRI Centre for Doctoral Training in Application of Artificial Intelligence to the study of Environmental Risks (AI4ER) (EP/S022961/1).
Note
The authors acknowledge the authors of the Detectron2 package which provides the Mask R-CNN architecture.
Install and load libraries¶
## install detectron2
!python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
!pip -q install numpy==1.20.0
## install geospatial libraries
!pip -q install geopandas==0.10.1
!pip -q install rasterio==1.2.9
!pip -q install fiona==1.8.18
!pip -q install shapely==1.7.1
## install interactive plotting
!pip -q install geoviews
Collecting git+https://github.com/facebookresearch/detectron2.git
Cloning https://github.com/facebookresearch/detectron2.git to /tmp/pip-req-build-mqwyhxup
Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/detectron2.git /tmp/pip-req-build-mqwyhxup
Resolved https://github.com/facebookresearch/detectron2.git to commit 0703e08a5f589f7503a3fbfce41309c80204eec8
Preparing metadata (setup.py) ... ?25l- done
?25hRequirement already satisfied: Pillow>=7.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (9.2.0)
Requirement already satisfied: matplotlib in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (3.5.3)
Collecting pycocotools>=2.0.2
Downloading pycocotools-2.0.4.tar.gz (106 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 29.1 MB/s eta 0:00:00
?25h Installing build dependencies ... ?25l- \ | / - \ done
?25h Getting requirements to build wheel ... ?25l- done
?25h Preparing metadata (pyproject.toml) ... ?25l- done
?25hCollecting termcolor>=1.1
Downloading termcolor-2.0.1-py3-none-any.whl (5.4 kB)
Collecting yacs>=0.1.8
Downloading yacs-0.1.8-py3-none-any.whl (14 kB)
Collecting tabulate
Downloading tabulate-0.8.10-py3-none-any.whl (29 kB)
Requirement already satisfied: cloudpickle in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (2.2.0)
Requirement already satisfied: tqdm>4.29.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (4.64.1)
Collecting tensorboard
Downloading tensorboard-2.10.0-py3-none-any.whl (5.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.9/5.9 MB 114.7 MB/s eta 0:00:00
?25hCollecting fvcore<0.1.6,>=0.1.5
Downloading fvcore-0.1.5.post20220512.tar.gz (50 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.1/50.1 kB 17.6 MB/s eta 0:00:00
?25h Preparing metadata (setup.py) ... ?25l- done
?25hCollecting iopath<0.1.10,>=0.1.7
Downloading iopath-0.1.9-py3-none-any.whl (27 kB)
Requirement already satisfied: future in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (0.18.2)
Collecting pydot
Downloading pydot-1.4.2-py2.py3-none-any.whl (21 kB)
Collecting omegaconf>=2.1
Downloading omegaconf-2.2.3-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.3/79.3 kB 27.3 MB/s eta 0:00:00
?25hCollecting hydra-core>=1.1
Downloading hydra_core-1.2.0-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.1/151.1 kB 46.8 MB/s eta 0:00:00
?25hCollecting black==22.3.0
Downloading black-22.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 101.1 MB/s eta 0:00:00
?25hCollecting timm
Downloading timm-0.6.7-py3-none-any.whl (509 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 510.0/510.0 kB 89.9 MB/s eta 0:00:00
?25hCollecting fairscale
Downloading fairscale-0.4.9.tar.gz (265 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 265.2/265.2 kB 69.1 MB/s eta 0:00:00
?25h Installing build dependencies ... ?25l- \ | done
?25h Getting requirements to build wheel ... ?25ldone
?25h Installing backend dependencies ... ?25l- \ done
?25h Preparing metadata (pyproject.toml) ... ?25l- done
?25hRequirement already satisfied: packaging in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from detectron2==0.6) (21.3)
Requirement already satisfied: click>=8.0.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from black==22.3.0->detectron2==0.6) (8.1.3)
Collecting pathspec>=0.9.0
Downloading pathspec-0.10.1-py3-none-any.whl (27 kB)
Requirement already satisfied: typing-extensions>=3.10.0.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from black==22.3.0->detectron2==0.6) (4.3.0)
Collecting mypy-extensions>=0.4.3
Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting tomli>=1.1.0
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting platformdirs>=2
Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
Requirement already satisfied: numpy in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6) (1.22.4)
Requirement already satisfied: pyyaml>=5.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6) (6.0)
Requirement already satisfied: importlib-resources in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from hydra-core>=1.1->detectron2==0.6) (5.9.0)
Collecting antlr4-python3-runtime==4.9.*
Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.0/117.0 kB 43.1 MB/s eta 0:00:00
?25h Preparing metadata (setup.py) ... ?25l- done
?25hCollecting portalocker
Downloading portalocker-2.5.1-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from matplotlib->detectron2==0.6) (3.0.9)
Requirement already satisfied: fonttools>=4.22.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from matplotlib->detectron2==0.6) (4.37.2)
Requirement already satisfied: python-dateutil>=2.7 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from matplotlib->detectron2==0.6) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from matplotlib->detectron2==0.6) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from matplotlib->detectron2==0.6) (1.4.4)
Requirement already satisfied: torch>=1.8.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from fairscale->detectron2==0.6) (1.8.0.post3)
Collecting google-auth-oauthlib<0.5,>=0.4.1
Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting tensorboard-plugin-wit>=1.6.0
Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 781.3/781.3 kB 100.3 MB/s eta 0:00:00
?25hCollecting google-auth<3,>=1.6.3
Downloading google_auth-2.11.0-py2.py3-none-any.whl (167 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.2/167.2 kB 52.7 MB/s eta 0:00:00
?25hCollecting tensorboard-data-server<0.7.0,>=0.6.0
Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 126.4 MB/s eta 0:00:00
?25hCollecting grpcio>=1.24.3
Downloading grpcio-1.49.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 117.8 MB/s eta 0:00:00
?25hRequirement already satisfied: requests<3,>=2.21.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from tensorboard->detectron2==0.6) (2.28.1)
Requirement already satisfied: setuptools>=41.0.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from tensorboard->detectron2==0.6) (65.3.0)
Collecting protobuf<3.20,>=3.9.2
Downloading protobuf-3.19.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 113.8 MB/s eta 0:00:00
?25hCollecting absl-py>=0.4
Downloading absl_py-1.2.0-py3-none-any.whl (123 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.4/123.4 kB 41.0 MB/s eta 0:00:00
?25hRequirement already satisfied: markdown>=2.6.8 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from tensorboard->detectron2==0.6) (3.4.1)
Requirement already satisfied: wheel>=0.26 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from tensorboard->detectron2==0.6) (0.37.1)
Collecting werkzeug>=1.0.1
Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 232.7/232.7 kB 62.7 MB/s eta 0:00:00
?25hRequirement already satisfied: torchvision in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from timm->detectron2==0.6) (0.9.0a0)
Collecting rsa<5,>=3.1.4
Downloading rsa-4.9-py3-none-any.whl (34 kB)
Requirement already satisfied: six>=1.9.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6) (1.16.0)
Collecting cachetools<6.0,>=2.0.0
Downloading cachetools-5.2.0-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1
Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 51.2 MB/s eta 0:00:00
?25hCollecting requests-oauthlib>=0.7.0
Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard->detectron2==0.6) (4.11.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6) (2022.9.14)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6) (1.26.11)
Requirement already satisfied: idna<4,>=2.5 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6) (3.3)
Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from werkzeug>=1.0.1->tensorboard->detectron2==0.6) (2.1.1)
Requirement already satisfied: zipp>=3.1.0 in /usr/share/miniconda/envs/repo/lib/python3.8/site-packages (from importlib-resources->hydra-core>=1.1->detectron2==0.6) (3.8.1)
Collecting pyasn1<0.5.0,>=0.4.6
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 27.5 MB/s eta 0:00:00
?25hCollecting oauthlib>=3.0.0
Downloading oauthlib-3.2.1-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 55.0 MB/s eta 0:00:00
?25hBuilding wheels for collected packages: detectron2, fvcore, antlr4-python3-runtime, pycocotools, fairscale
Building wheel for detectron2 (setup.py) ... ?25l- \ | / - \ | / - done
?25h Created wheel for detectron2: filename=detectron2-0.6-cp38-cp38-linux_x86_64.whl size=890522 sha256=18cc2f7c0832034d7f37bb06d46c8d0d01d6fc6d611799d67db0e32a92d103b0
Stored in directory: /tmp/pip-ephem-wheel-cache-sotzf_0v/wheels/19/ac/65/e48e5e4ec2702274d927c5a6efb75709b24014371d3bb778f2
Building wheel for fvcore (setup.py) ... ?25l- done
?25h Created wheel for fvcore: filename=fvcore-0.1.5.post20220512-py3-none-any.whl size=61263 sha256=1701317df65bda30d20d2d0304aa6fa7544d9eddf4fda392f73928d1cf638b82
Stored in directory: /home/runner/.cache/pip/wheels/bc/f4/d9/8b3c3f254c28aa2daf5e2f5a8070b0a960278733fd2eb1f7a2
Building wheel for antlr4-python3-runtime (setup.py) ... ?25l- \ done
?25h Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144554 sha256=64719a03a0b12fedf37dae026b0022abb4c19189874236e23d67ca4c9e1634cd
Stored in directory: /home/runner/.cache/pip/wheels/b1/a3/c2/6df046c09459b73cc9bb6c4401b0be6c47048baf9a1617c485
Building wheel for pycocotools (pyproject.toml) ... ?25l- \ | done
?25h Created wheel for pycocotools: filename=pycocotools-2.0.4-cp38-cp38-linux_x86_64.whl size=103694 sha256=8f2bfe62ea55547b81de18eb200323b29cc186e5256adaf4a9829a89f83078df
Stored in directory: /home/runner/.cache/pip/wheels/dd/e2/43/3e93cd653b3346b3d702bb0509bc611189f95d60407bff1484
Building wheel for fairscale (pyproject.toml) ... ?25l- \ done
?25h Created wheel for fairscale: filename=fairscale-0.4.9-py3-none-any.whl size=327402 sha256=7ea1955a7376034fe8a58958eb3ce66d277b344d696115a08fba9c3769440061
Stored in directory: /home/runner/.cache/pip/wheels/0e/f6/e0/52eeae3f7d5da08731b593f735b064c52c5a8bd63361d47ecf
Successfully built detectron2 fvcore antlr4-python3-runtime pycocotools fairscale
Installing collected packages: tensorboard-plugin-wit, pyasn1, mypy-extensions, antlr4-python3-runtime, yacs, werkzeug, tomli, termcolor, tensorboard-data-server, tabulate, rsa, pydot, pyasn1-modules, protobuf, portalocker, platformdirs, pathspec, omegaconf, oauthlib, grpcio, cachetools, absl-py, requests-oauthlib, iopath, hydra-core, google-auth, fairscale, black, timm, pycocotools, google-auth-oauthlib, fvcore, tensorboard, detectron2
Successfully installed absl-py-1.2.0 antlr4-python3-runtime-4.9.3 black-22.3.0 cachetools-5.2.0 detectron2-0.6 fairscale-0.4.9 fvcore-0.1.5.post20220512 google-auth-2.11.0 google-auth-oauthlib-0.4.6 grpcio-1.49.0 hydra-core-1.2.0 iopath-0.1.9 mypy-extensions-0.4.3 oauthlib-3.2.1 omegaconf-2.2.3 pathspec-0.10.1 platformdirs-2.5.2 portalocker-2.5.1 protobuf-3.19.5 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycocotools-2.0.4 pydot-1.4.2 requests-oauthlib-1.3.1 rsa-4.9 tabulate-0.8.10 tensorboard-2.10.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 termcolor-2.0.1 timm-0.6.7 tomli-2.0.1 werkzeug-2.2.2 yacs-0.1.8
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
fairscale 0.4.9 requires numpy>=1.22.0, but you have numpy 1.20.0 which is incompatible.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
black 22.3.0 requires click>=8.0.0, but you have click 7.1.2 which is incompatible.
import cv2
from PIL import Image
import os
import numpy as np
import urllib.request
import glob
# intake library and plugin
import intake
from intake_zenodo_fetcher import download_zenodo_files_for_entry
# geospatial libraries
import geopandas as gpd
from rasterio.transform import from_origin
import rasterio.features
import fiona
from shapely.geometry import shape, mapping, box
from shapely.geometry.multipolygon import MultiPolygon
# machine learning libraries
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.utils.visualizer import Visualizer, ColorMode
from detectron2.config import get_cfg
from detectron2.engine import DefaultTrainer
# visualisation
import holoviews as hv
import geoviews.tile_sources as gts
import matplotlib.pyplot as plt
import hvplot.pandas
import hvplot.xarray
import warnings
warnings.filterwarnings(action='ignore')
hv.extension('bokeh', width=100)
WARNING:root:Pytorch pre-release version 1.8.0.post3 - assuming intent to test it
WARNING:root:Pytorch pre-release version 1.8.0.post3 - assuming intent to test it
WARNING:root:Pytorch pre-release version 1.8.0.post3 - assuming intent to test it
WARNING:root:Torch AMP is not available on this platform
Set folder structure¶
# Define the project main folder
notebook_folder = './forest-modelling-treecrown_detectreeRGB'
# Set the folder structure
config = {
'in_geotiff': os.path.join(notebook_folder, 'input','tiff'),
'in_png': os.path.join(notebook_folder, 'input','png'),
'model': os.path.join(notebook_folder, 'model'),
'out_geotiff': os.path.join(notebook_folder, 'output','raster'),
'out_shapefile': os.path.join(notebook_folder, 'output','vector'),
}
# List comprehension for the folder structure code
[os.makedirs(val) for key, val in config.items() if not os.path.exists(val)]
[None, None, None, None, None]
Load and prepare input image¶
Fetch a GeoTIFF file of aerial forest imagery using intake¶
Let’s fetch a sample aerial image from a Zenodo repository.
# write a catalog YAML file
catalog_file = os.path.join(notebook_folder, 'catalog.yaml')
with open(catalog_file, 'w') as f:
f.write('''
sources:
sepilok_rgb:
driver: rasterio
description: 'NERC RGB images of Sepilok, Sabah, Malaysia (collection)'
metadata:
zenodo_doi: "10.5281/zenodo.5494629"
args:
urlpath: "{{ CATALOG_DIR }}/input/tiff/Sep_2014_RGB_602500_646600.tif"
''')
cat_tc = intake.open_catalog(catalog_file)
for catalog_entry in list(cat_tc):
download_zenodo_files_for_entry(
cat_tc[catalog_entry],
force_download=False
)
will download https://zenodo.org/api/files/271e78b4-b605-4731-a127-bd097e639bf8/Sep_2014_RGB_602500_646600.tif to /home/runner/work/forest-modelling-treecrown_detectreeRGB/forest-modelling-treecrown_detectreeRGB/forest-modelling-treecrown_detectreeRGB/input/tiff/Sep_2014_RGB_602500_646600.tif
tc_rgb = cat_tc["sepilok_rgb"].to_dask()
Inspect the aerial image¶
Let’s investigate the data-array, what is the shape? Bounds? Bands? CRS?
print('shape =', tc_rgb.shape,',', 'and number of bands =', tc_rgb.count, ', crs =', tc_rgb.crs)
shape = (4, 1400, 1400) , and number of bands = <bound method ImplementsArrayReduce._reduce_method.<locals>.wrapped_func of <xarray.DataArray (band: 4, y: 1400, x: 1400)>
array([[[36166.285 , 34107.22 , ..., 20260.998 , 11166.631 ],
[32514.84 , 28165.994 , ..., 24376.36 , 21131.947 ],
...,
[15429.493 , 16034.794 , ..., 19893.691 , 19647.646 ],
[12534.722 , 14003.215 , ..., 21438.908 , 22092.525 ]],
[[38177.168 , 36530.74 , ..., 19060.268 , 11169.006 ],
[34625.227 , 30270.379 , ..., 21760.09 , 20796.621 ],
...,
[17757.678 , 16818.102 , ..., 22538.023 , 23093.508 ],
[13403.302 , 13354.489 , ..., 24638.21 , 25545.938 ]],
[[13849.501 , 14158.603 , ..., 9385.764 , 7401.662 ],
[13252.31 , 13373.801 , ..., 12217.845 , 10666.252 ],
...,
[13471.741 , 11533.697 , ..., 7536.6924, 8397.009 ],
[13724.59 , 11057.722 , ..., 9778.8125, 11174.72 ]],
[[65535. , 65535. , ..., 65535. , 65535. ],
[65535. , 65535. , ..., 65535. , 65535. ],
...,
[65535. , 65535. , ..., 65535. , 65535. ],
[65535. , 65535. , ..., 65535. , 65535. ]]], dtype=float32)
Coordinates:
* band (band) int64 1 2 3 4
* y (y) float64 6.467e+05 6.467e+05 6.467e+05 ... 6.466e+05 6.466e+05
* x (x) float64 6.025e+05 6.025e+05 6.025e+05 ... 6.026e+05 6.026e+05
Attributes:
transform: (0.1, 0.0, 602480.0, 0.0, -0.1, 646720.0)
crs: +init=epsg:32650
res: (0.1, 0.1)
is_tiled: 0
nodatavals: (-3.3999999521443642e+38, -3.3999999521443642e+38, -3.399...
scales: (1.0, 1.0, 1.0, 1.0)
offsets: (0.0, 0.0, 0.0, 0.0)
AREA_OR_POINT: Area> , crs = +init=epsg:32650
Save the RGB bands of the GeoTIFF file as a PNG¶
Mask R-CNN requires images in png format. Let’s export the RGB bands to a png file.
minx = 602500
miny = 646600
R = tc_rgb[0]
G = tc_rgb[1]
B = tc_rgb[2]
# stack up the bands in an order appropriate for saving with cv2, then rescale to the correct 0-255 range for cv2
# you will have to change the rescaling depending on the values of your tiff!
rgb = np.dstack((R,G,B)) # BGR for cv2
rgb_rescaled = 255*rgb/65535 # scale to image
# save this as png, named with the origin of the specific tile - change the filepath!
filepath = config['in_png'] + '/' + 'tile_' + str(minx) + '_' + str(miny) + '.png'
cv2.imwrite(filepath, rgb_rescaled)
True
Read in and display the PNG file¶
im = cv2.imread(filepath)
plot_input = plt.figure(figsize=(15,15))
plt.imshow(Image.fromarray(im))
plt.title('Input image',fontsize='xx-large')
plt.axis('off')
plt.show()
Modelling¶
Download the pretrained model¶
# define the URL to retrieve the model
fn = 'model_final.pth'
url = f'https://zenodo.org/record/5515408/files/{fn}?download=1'
urllib.request.urlretrieve(url, config['model'] + '/' + fn)
('./forest-modelling-treecrown_detectreeRGB/model/model_final.pth',
<http.client.HTTPMessage at 0x7fd21a95f850>)
Settings of detectron2 config¶
The following lines allow configuring the main settings for predictions and load them into a DefaultPredictor object.
cfg = get_cfg()
# if you want to make predictions using a CPU, run the following line. If using GPU, hash it out.
cfg.MODEL.DEVICE='cpu'
# model and hyperparameter selection
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml"))
cfg.DATALOADER.NUM_WORKERS = 2
cfg.SOLVER.IMS_PER_BATCH = 2
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1
### path to the saved pre-trained model weights
cfg.MODEL.WEIGHTS = config['model'] + '/model_final.pth'
# set confidence threshold at which we predict
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.15
#### Settings for predictions using detectron config
predictor = DefaultPredictor(cfg)
Outputs¶
Showing the predictions from detectreeRGB¶
outputs = predictor(im)
v = Visualizer(im[:, :, ::-1], scale=1.5, instance_mode=ColorMode.IMAGE_BW) # remove the colors of unsegmented pixels
v = v.draw_instance_predictions(outputs["instances"].to("cpu"))
image = cv2.cvtColor(v.get_image()[:, :, :], cv2.COLOR_BGR2RGB)
plot_predictions = plt.figure(figsize=(15,15))
plt.imshow(Image.fromarray(image))
plt.title('Predictions',fontsize='xx-large')
plt.axis('off')
plt.show()
Convert predictions into geospatial files¶
To GeoTIFF¶
mask_array = outputs['instances'].pred_masks.cpu().numpy()
# get confidence scores too
mask_array_scores = outputs['instances'].scores.cpu().numpy()
num_instances = mask_array.shape[0]
mask_array_instance = []
output = np.zeros_like(mask_array)
mask_array_instance.append(mask_array)
output = np.where(mask_array_instance[0] == True, 255, output)
fresh_output = output.astype(np.float)
x_scaling = 140/fresh_output.shape[1]
y_scaling = 140/fresh_output.shape[2]
# this is an affine transform. This needs to be altered significantly.
transform = from_origin(int(filepath[-17:-11])-20, int(filepath[-10:-4])+120, y_scaling, x_scaling)
output_raster = config['out_geotiff'] + '/' + 'predicted_rasters_' + filepath[-17:-4]+ '.tif'
new_dataset = rasterio.open(output_raster, 'w', driver='GTiff',
height = fresh_output.shape[1], width = fresh_output.shape[2], count = fresh_output.shape[0],
dtype=str(fresh_output.dtype),
crs='+proj=utm +zone=50 +datum=WGS84 +units=m +no_defs',
transform=transform)
new_dataset.write(fresh_output)
new_dataset.close()
To shapefile¶
# Read input band with Rasterio
with rasterio.open(output_raster) as src:
shp_schema = {'geometry': 'MultiPolygon','properties': {'pixelvalue': 'int', 'score': 'float'}}
crs = src.crs
for i in range(src.count):
src_band = src.read(i+1)
src_band = np.float32(src_band)
conf = mask_array_scores[i]
# Keep track of unique pixel values in the input band
unique_values = np.unique(src_band)
# Polygonize with Rasterio. `shapes()` returns an iterable
# of (geom, value) as tuples
shapes = list(rasterio.features.shapes(src_band, transform=src.transform))
if i == 0:
with fiona.open(config['out_shapefile'] + '/predicted_polygons_' + filepath[-17:-4] + '_' + str(0) + '.shp', 'w', 'ESRI Shapefile',
shp_schema) as shp:
polygons = [shape(geom) for geom, value in shapes if value == 255.0]
multipolygon = MultiPolygon(polygons)
# simplify not needed here
#multipolygon = multipolygon_a.simplify(0.1, preserve_topology=False)
shp.write({
'geometry': mapping(multipolygon),
'properties': {'pixelvalue': int(unique_values[1]), 'score': float(conf)}
})
else:
with fiona.open(config['out_shapefile'] + '/predicted_polygons_' + filepath[-17:-4] + '_' + str(0)+'.shp', 'a', 'ESRI Shapefile',
shp_schema) as shp:
polygons = [shape(geom) for geom, value in shapes if value == 255.0]
multipolygon = MultiPolygon(polygons)
# simplify not needed here
#multipolygon = multipolygon_a.simplify(0.1, preserve_topology=False)
shp.write({
'geometry': mapping(multipolygon),
'properties': {'pixelvalue': int(unique_values[1]), 'score': float(conf)}
})
Interactive map to visualise the exported files¶
Plot tree crown delineation predictions and scores¶
# load and plot polygons
in_shp = glob.glob(config['out_shapefile'] + '/*.shp')
poly_df = gpd.read_file(in_shp[0])
plot_vector = poly_df.hvplot(hover_cols=['score'], legend=False).opts(fill_color=None,line_color=None,alpha=0.5, width=800, height=600, xaxis=None, yaxis=None)
plot_vector
Plot the exported GeoTIFF file¶
# load and plot RGB image
r = tc_rgb.sel(band=[1,2,3])
normalized = r/(r.quantile(.99,skipna=True)/255)
mask = normalized.where(normalized < 255)
int_arr = mask.astype(int)
plot_rgb = int_arr.astype('uint8').hvplot.rgb(
x='x', y='y', bands='band', data_aspect=0.8, hover=False, legend=False, rasterize=True, xaxis=None, yaxis=None, title='Tree crown delineation by detectreeRGB'
)
Overlay the prediction labels and image¶
Note we have some artifacts in the RGB image due to the transformations using the normalization procedure.
plot_predictions_interactive = plot_rgb * plot_vector
plot_predictions_interactive
OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Save plot¶
hvplot.save(plot_predictions_interactive, notebook_folder + '/interactive_predictions.html')
Summary¶
We have read in a raster, chosen a tile and made predictions on it. These predictions can then be transformed to shapefiles and examined in GIS software!
We made the predictions on the
pngusing a pretrained Mask R-CNN model,detectreeRGB.The outputs showed the model capability to detect and delineate tree crowns from aerial imagery.
We then extracted our predictions, added the geospatial location back in, and exported them as shapefiles including the confidence score assigned to each prediction by the model.
Visualised the predictions on an interactive map!
Citing this Notebook¶
Sebastian H. M. Hickman, and Alejandro Coca-Castro. “Tree crown delineation using detectreeRGB (Jupyter Notebook) published in the Environmental Data Science book.” ROHub. Mar 27 ,2022. https://w3id.org/ro-id/94486a7f-e046-461f-bbb9-334ec7b57040.
Additional information¶
Codebase: version 1.0.0 with commit 16a5a1c
License: The code in this notebook is licensed under the MIT License. The Environmental Data Science book is licensed under the Creative Commons by Attribution 4.0 license. See further details here.
Contact: If you have any suggestion or report an issue with this notebook, feel free to create an issue or send a direct message to environmental.ds.book@gmail.com.
Last tested: 2022-09-16